With the advent of the Internet, users are increasingly sharing images with one another. Often, these images are shared through social networks, personal web pages, or image search services that allow users to share pictures. Because the web sites offering these images often store a vast number of images, mechanisms for searching for and retrieving images have been developed. One such mechanism utilizes low level features of the images themselves, categorizing images by their low level features and associating the features with searchable descriptors. Another mechanism utilizes image tags, such as image descriptors provided by users. These tags often include terms associated with the content of an image, such as “dog” for a picture of a dog. Tags also include other types of descriptors, such as a verb describing what is happening in a picture (e.g., “jumping”), an adjective (e.g., “beautiful”), or a term meaningful only to the user doing the tagging (e.g., a name of the dog).
Often, though, tags only describe a small part of an image, leading to image results that do not match user expectations. For example, a user may search for a balloon, hoping to find a picture of a large, blue balloon centered within the image. Most of the results received by the user will not match this expectation. Results might include any number of pictures where the balloons are difficult to see, out of the center of view, or have a different color. Even if the user were to search for “centered blue balloon,” the results would not be substantially better. The reason for this is that there is a disconnect between tagging behavior and searching behavior. In tagging images, users tend to apply a single tag for each object in the image (e.g., balloon) without describing the attributes of the object (e.g., where the object is located, its color, its size, etc.). In contrast, users searching for an image tend to be looking for highly specific search results. Without tags describing these additional specifics, however, a tag-based search will return many images that do not meet user expectations.